Weak agreement between Antivirogram and Phenosense assays in predicting reduced susceptibility to antiretroviral drugs.

Qari and colleagues (3) in 2002 reported a comparative analysis of two commercial phenotypic assays (Antivirogram and PhenoSense) for human immunodeficiency virus type 1 (HIV-1) drug resistance testing. The Antivirogram (1) and PhenoSense (2) assays are both recombinant virus-based assays to measure the drug susceptibilities of patient-derived HIV protease or reverse transcriptase sequences and were developed by Virco (Belgium) and ViroLogic, Inc. (South San Francisco, Calif.), respectively. In the paper, the authors collected viruses from 50 clinical specimens and measured their susceptibility to 12 to 15 antiretroviral drugs using both assays. Based on 529 pairs of observations, they reported a 91.5% concordance by binary classification of sensitive versus reduced susceptibility. They concluded that “the Antivirogram and PhenoSense assays correlate well, despite the use of different testing strategies.” Because of the lack of quantitative data in their paper on the actual correlation coefficients for the two phenotypic assays, we downloaded resistance estimates (increases [n-fold] in 50% inhibitory concentrations relative to that for the wild type) for the samples used by Qari et al. (3) from the Stanford HIV drug resistance database (4). Despite the high binary concordance obtained with the samples that they tested, correlations between the resistance values were surprisingly poor for many of the drugs (Table 1). For some drugs, such as delavirdine (DLV), the two assays had high Pearson correlation coefficients, but these high correlations were due to a small number of highly resistant samples. We have focused, therefore, on Spearman rank correlation coefficients (rs) since they are less sensitive to extreme values. For the 14 drugs, rs values ranged from 0.018 for saquinavir (SQV) to 0.551 for lamivudine (3TC), with a median of 0.290. Spearman’s rank correlation test is a statistical test in which the null hypothesis is that two variables are uncorrelated. Performing a separate test for each drug, we failed to reject this null hypothesis for eight drugs using a significance value of 0.05 (Table 1). Furthermore, from Table 1 of the original paper, we noticed that only a small fraction of the samples were classified as having “reduced susceptibility” by both assays. We argue that because of the high fraction of drug-susceptible viruses in their study, their overall 91.5% concordance result is somewhat misleading. (Given frequencies of susceptible viruses of 89.6 and 91.7% for the Antivirogram and PhenoSense assays, respectively, one would expect 83.0% concordance by chance). In clinical settings it is most important to have high prediction accuracy when the virus has reduced susceptibility to drugs. This argues for a closer examination of samples predicted to have reduced susceptibility on the basis of Qari et al.’s cutoff values (3). Of the 44 samples that are classified as having reduced susceptibility by the Antivirogram assay, 17 are reported as sensitive by the PhenoSense assay, resulting in a concordance of only 61.4% (95% confidence interval [CI], 45.5 to 75.6%). Of the 55 samples that are classified as having reduced susceptibility by the PhenoSense assay, 28 are reported as sensitive by the Antivirogram assay, resulting in a concordance of only 49.1% (95% CI, 35.4 to 62.9%). We conclude that resistance estimates from the Antivirogram and PhenoSense assays do not correlate well for the data shown by Qari et al. (3). Although these assays may correlate better for data sets that include more resistant viruses, in the absence of data showing that one test is better than the other, we suggest that classifications of reduced susceptibilities resulting from these phenotypic assays be interpreted with caution.